We introduce a model that learns to convert simple hand drawings into graphics programs written in a subset of \LaTeX.~The

Summary The paper presents a two-stage method to infer programs that generate input drawings. Initially, a CNN architecture is applied to translate the drawing into a sequence of primitive draw commands (lines, rectangles, circles, etc). Then, a program synthesizer is used to collapse the straight-line program into a proper program with conditionals and loops. First, let me say that I absolutely love the results of this paper. It is a wonderful combination of relatively simple tools to achieve something unexpected.

We introduce a model that learns to convert simple hand drawings into graphics programs written in a subset of \LaTeX. The model combines techniques from deep learning and program synthesis. We learn a convolutional neural network that proposes plausible drawing primitives that explain an image. These drawing primitives are a specification (spec) of what the graphics program needs to draw. We learn a model that uses program synthesis techniques to recover a graphics program from that spec.